The relationship between HRR-based similarity and similarity based on structural kernels

Work in machine learning on kernel-based methods over discrete structures, such as strings and trees, uses a variety of kernels to measure similarity between structures (Haussler, 1999, Collins and Duffy, 2002, Bod, 1998). For example, a kernel for strings could count the number of matching substrings, and kernel for trees could count the number of matching subtrees. A kernel is always a dot product between two feature vectors, i.e., a function K where K(x, y) = h(x) · h(y) = ∑ i hi(x)hi(y). The function h is some mapping of structures onto numerical vectors – hi(x) is the value of the ith feature of structure x. For example, in a kernel for strings, hi(x) could be the number of times the ith substring (in some enumeration of all substrings in the data set of interest) occurs in string x. What makes kernel-based methods practical is that there are efficient methods for computing K(x, y) that do not require h(x) (or h(y)) to ever be explicitly represented or even enumerated. This is fortunate, as the length of h-vectors can be exponential in the size of the structures being represented – even the number of non-zero elements in h(x) can be exponential in the size of x. These efficient algorithms are typically based on dynamicprogramming techniques and calculate K(x, y) in time polynomial in the size of x and y. For some kernels there are even algorithms with quadratic time and linear time complexity (e.g., Vishwanathan and Smola, 2003, Lodhi et al., 2000). Kernel-based machine learning algorithms have shown some degree of success in challenging tasks, e.g., text classification (Lodhi et al., 2000), and natural language parsing (Collins and Duffy, 2002). Hence it is interesting to see how